Enabling Reinforcement Learning for Open Dialogue Systems through Speech Stress Detection

The human speech signal contains a wide range of paralinguistic information and the interpretation and exploitation of this information presents a fascinating challenge. This paper seeks to demonstrate that emotion is central to the construction of an openended dialogue system. Emotion forms a useful, practical, metric that enables a agent to both maintain a user’s positive emotional state and allow it to judge and refine its current dialogue strategy. One way to determine the emotional state of the user is through an interpretation of the speech signal. Accordingly, this paper details a method of detecting emotion within speech and then exploiting this information to drive an open-ended dialogue system. After presenting the outline of an advanced open-ended dialogue system we test an initial model to judge the validity of this approach. This simplified model perceives a number of real speech utterances with varying emotional content (ranging from stressed to happy) and learns to manipulate the emotional state of an artificial user through reinforcement learning. The agent acquires the users emotional response to its replies and is able, through a balancing of exploration and exploitation, to maintain the users positive disposition. This initial work is encouraging and clearly shows that emotion can motivate open-ended dialogue. However, a number of substantial challenges remain, including the perception of natural, as opposed to acted, speech and the development of an unsupervised learning approach.